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Abstract 

Heteroskedasticity is a common feature of financial time series and is commonly addressed in the 
model building process through the use of ARCH and GARCH processes. More recently multivariate 
variants of these processes have been in the focus of research with attention given to methods seeking an 
efficient and economic estimation of a large number of model parameters. Due to the need for estimation of 
many parameters, however, these models may not be suitable for modeling now prevalent high-frequency 
volatility data. One potentially useful way to bypass these issues is to take a functional approach. In 
this paper, theory is developed for a new functional version of the generalized autoregressive condition¬ 
ally heteroskedastic process, termed fGARCH. The main results are concerned with the structure of the 
fGARCH(l,l) process, providing criteria for the existence of strictly stationary solutions both in the space 
of square-integrable and continuous functions. An estimation procedure is introduced and its consistency 
and asymptotic normality verified. A small empirical study highlights potential applications to intraday 
volatility estimation. 
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1 Introduction 

Modeling volatility is one of the prime objeetives of finaneial time series analysis. Research in the area 

has increased rapidly since the seminal contributions by Engle (1982), who introduced the autoregressive 

conditional heteroskedastic, ARCH, model, and Bollerslev (1986), who introduced the generalized ARCH, 

GARCH, model to deal with non-constant and randomly changing volatilities for univariate time series. 

Methodology based on these processes and their modifications have become major theoretical and applied 

tools to analyze stock returns, exchange rates and more. The book Francq and Zakoian (2010) provides an 
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excellent introduction to volatility processes and their applications. The interested reader may find detailed 
accounts on theoretical properties such as conditions for the existence of stationary solutions and finiteness of 
moments as well as methodology for the estimation of model parameters. 

With ever more data being collected in today’s financial sector, fhere has come a need to address new 
phenomena observed when dealing wifh high-frequency, infra-day observafions. While mulfivariafe volafilify 
processes such as fhe ones of Engle (2002) and Engle and Kroner (1995) mighf be used fo model high- 
frequency refurns, fhese models are fypically associafed wifh fhe difficull fask of esfimafing a large number 
of paramefers (see Chapfer 11 of Erancq and Zakoian, 2010). Surveys of mulfivariafe GARCH models and 
related volafilify processes may be found in Bauwens ef al. (2006) and Silvennoinen and Terasvirfa (2009). 
To alleviate fhe numerical burden of fhe paramefer esfimafion, fhe variance fargefing mefhod of Engle and 
Mezrich (1996) can be used, which was recenfly extended fo cover mulfivariafe GARCH models by Erancq 
el al. (2011, 2015). These mefhods, however, may sfill fail to produce salisfaclory resulfs in a high-frequency 
selling. 

Modeling dala as a colleclion of functions was popularized Ihrough fhe work of Ramsay and Silvermann 
(2005). While Ibis approach has now slarled fo become relevanl for fhe analysis of high-frequency volafilify 
dala, much of fhe research in fhe area has so far been devoted to homoskedaslic funclional lime series. The 
books by Bosq (2010) and Horvalh and Kokoszka (2012) provide an accounl of fhe slale-of-lhe-arl research on 
dependenl sequences of random functions, including functional auloregressive processes. More recenfly, pre- 
dicfion of funclional lime series was considered in Aue el al. (2015), while esfimafion in dependenl funclional 
linear models was discussed in Aue el al. (2014). 

A firsl slep to inlroduce a funclional heleroskedasfic framework in fhe Iradilion of Engle (1982) was un- 
derlaken in Hermann el al. (2013). These aulhors found conditions for fhe exislence of functional ARCH(l) 
processes, for which fhe conditional variance depends on fhe whole (infra-day) palh of fhe previous obser- 
valion. In fhe spirif of Bollerslev (1986), Ihis paper inlroduces a funclional model in which fhe condilional 
volafilify function of fhe presenl observalion is given as a functional linear combination of fhe (infra-day) 
palhs of bolh fhe pasl squared observation and ifs volafilify function. The following definilion is cenlral. 

Definition 1.1, A sequence of random functions {yi: i £ Z) is called afunctional GARCH process of orders 
(1,1), abbreviated by fGARCH( 1,1), if it satisfies the equations 


Vi — 

(1.1) 

= 6 + ayf_i + 

(1.2) 


where 6 is a nonnegative function and the integral operators a and (3 map nonnegative functions to nonnega- 
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tive functions, which is in short notation referred to as 


> 0, a > 0, /3 > 0. 


(1.3) 


Further, the innovations 

{si'. i € Z) are independent and identically distributed random functions. 


(1.4) 


Additional assumptions will be introduced were needed. Note that the integral operators a and /3 are, for 
t G [0,1], defined by {ax){t) = f a(t, s)x(s)ds and (/3x)(t) = f /3(t, s)x(s)ds, where x is an arbitrary 
element of Z^[0,1] and f is used to mean . The integral kernel functions a(s, t) and /3(s, t) are elements 
of the Hilbert space Z^[0,1]^ and consequently bounded. 

The model given by O) and exhibits two time variables. The first is labeled by the integer i 
and will in the present context often refer to trading day i, even though other time units are possible. The 
second time variable is labeled by the real-valued t which, without loss of generality, takes values in the unit 
interval [0,1]. With t, intra-day trading time is parameterized. From this set-up, it should become clear that a 
modeling of intra-day seasonality may be accommodated by the fGARCH model. Doing this in a functional 
framework, may lead to efficiency gains over a multivariafe modeling approach if the underlying process is 
sufficiently smooth. Similar to the univariate case argued in Bollerslev (1986), ( |1.2[ ) should lead to a more 
flexible modeling of the functional volatility lag structure compared to the fARCH(l) process of Hermann 
et al. (2013). The focus is here on the first-order (1,1) case. Even in the univariate case, this is the most 
widely used GARCH process, used as default in the financial industry because it tends to work well (see, 
for example, the overview article by Zivot, 2009). Similar arguments should also apply in the more complex 
functional world. 

The paper proceeds as follows. Theoretical properties of the fGARCH(l,l) process defined via ( |1.1[ ) 
and ( |1.2[ ) are discussed in Section This section contains the main contributions of this work, establishing 
conditions for the existence of the functional stochastic difference equations B and ( |1.2| ). Since one may 
view the functions pi and af both as members of the space of square-integrable and continuous functions, two 
different theorems are stated to cover both cases. The results on the structure of the fGARCH(l,l) process 
are accompanied by additional results on the estimation of quantities appearing in ( |1.2| ). These are given 
in Section Empirical aspects are highlighted in Section Proofs of the theoretical results are given in 
Section |5] 

2 Structure 

This section discusses properties of the fGARCH(l,l) process, in particular notions of unique, strictly sta¬ 
tionary solutions to the defining equations B and ( |1.2| ). Two different settings are of interest: solutions 
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in = L^[0,1], the space of square integrable functions on [0,1], and in C[0,1], the space of continuous 
functions on [0,1]. In both cases, solutions can be characterized through properties of the random integral 
kernel 

7 o(f, s) = a{t, s)eo('S) + /3{t, s). 

By assumption, 7 o(s,t) is an element of L^[0,1]^ with probability one and, following Riesz and Sz.-Nagy 
(1990, p. 148), gives rise to the random integral operator 70 defined through {'yx){t) = f 7o(s, t)x{s)ds. Note 
that 7 o > 0 (it maps nonnegative functions to nonnegative functions) and denote its (random) Hilbert-Schmidt 
norm by 

hoh= (^JJ li{t,s)dtds^ . 

In the case and for the fGARCH(l,l), the Hilbert-Schmidt norm ||7o||s will play the role of the usual 
top Lyapunov exponent when characterizing the existence of stationary solutions to random difference equa¬ 
tions (see, for example, Boucherol and Picard, 1992). One may also observe the similarity to the univariate 
GARCH(1,1) case discussed in Nelson (1990). Additional information on properties of function spaces may 
be found in the monograph Bosq (2000). 


Theorem 2.1, Assume that ( 1.31 and ( |1.4| ) hold, that 6 G L^, and that eo ^ with probability one. 

(i)If 

-00 < E[log|| 7 o||s] < 0 , ( 2 . 1 ) 


then (O' and (O have a unique, strictly stationary and nonanticipative solution in L^. 

(ii) If there is v > such that 

E[||7o||s1 < 1, (2.2) 

then (O and (O have a unique, strictly stationary and nonanticipative solution in anr/E[||crQ||2] < 00. 


The proof of Theorem 2.1 is given in Section]^ If {pi: i ^ Z) is an fARCH(l) process, then 7o(s, t) = 
a(s, t)£Q{s). The condition for the existence of a strictly stationary solution for the fARCH(l) equations in 
used in Hermann et al. (2013) can be rewritten as E[||7o||g] < 1. It can then be seen readily from an 
application of Jensen’s inequality that this condition implies (|2.1|). 


Corollary 2.1, The following statements are consequences ofTheorem \2.1\ 
(i) There is afunctional g such that 


0"*^ = 9(G-i,G- 2, • • f G E, 


(2.3) 


and therefore {af : i G Z) and {yi\ i G Z) are Bernoulli shifts. 

(ii) The representation p.3[ ) implies the ergodicity of (af : i Glfj and (yi'. i G Z). 
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(Hi) Both (af : i G Z) and (y*: i G Z) are weakly dependent processes. 

(iv) For £ G N and i,j G Z, let ■ be independent copies of Eq and define 

2 _ ( (*) (i) \ 

■ ■ ■ ^i—£i ^i—1—2^1—£i ■ • ■)■ 


If (2.2 1 is satisfied, then there is a constant 0 < p < 1 such that 


^[\\<yl-<ylAV2]=0{p% 


(2.4) 


Statement (iv) of the corollary says that, for every I, the sequence {a~ : i ^ Z) can be approximated with 


the ^-dependent sequence : z G Z) under a geometric rate. Consequently, if (^1 holds with some v >2, 
then the partial sum process ~ satisfies the functional central limit theorem in 

the space of square integrable processes; see Berkes et al. (2012) and Jirak (2013) for recent theoretical 
contributions in this area. 

If smoother trajectories are required of the fGARCH(l,l) and its volatility process, one may view (y* : i G 
Z) and [af : z G Z) as functional sequences in C[0,1]. To give the analog of Theorem 2.1 for continuous 
functions, let 7 o = / 7o(') s)ds and observe that 70 G S[0,1]. The next theorem shows that the role of || 7 o||s 
for square integrable functions is assumed by || 7 o||e = sup^ l 7 o(f)| for continuous functions. 


Theorem 2.2. Assume that ( |1.3| ) and ( |1.4| ) hold, that 6 G C[0,1], and that sq G C[0, 1] with probability one. 

(i) If 

-00 < E[log||7o||e] < 0, (2.5) 

then O' and (O have a unique, strictly stationary and nonanticipative solution in C[ 0 , 1 ]. 

(ii) If there is v > D such that 

E[||7o||^]<l, (2.6) 

then (O and ( |1.2| ) have a unique, strictly stationary nonanticipative solution in S[0,1] and E[||cjQ||g] < 00 . 

Corollary 2.2. As in Corollary \2.1\ the sequences (pi'. z G Z) and (at: i ^ Z) are weakly dependent. Under 
condition (| 2 . 6 [) there is a constant 0 < p < 1 such that 


^[\Wf-alr^]=0{p^), 

where alg is defined in Corollary 

The proofs of Theorem |2.2| and Corollary |2.2| are given in Section]^ The next section covers an estimation 
procedure for the fGARCH(l,l) process. Empirical examples are part of Section]^ 


2.i 
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3 Estimation 


The goal of this section is to obtain an estimate of the volatility equation ( |1.2| ), that is, of the function S and the 
operators a and /3 in Q. Since the objects involved in the estimation procedure are infinite-dimensional, an 
M-dimensional class = {‘f 1 ,^ 2 , ■■■, ‘fu} of orthonormal functions on [0,1] is introduced to represent 6, 
a and /3 in the following way. It is assumed that S can be represented as a linear combination of the functions 
in that is, 

M 

5 = ^ ^ dm^m- (3-1) 

m=l 

It is further assumed that the integral kernels a(s, t) and /3(s, t) are elements of the span of x so that 

M M 

a(t,s)= ^ and f3{t,s)= ^ bm,m'^mit)^Pm'is). (3.2) 

m,m'=l m,m'=l 

With these requirements in place, the problem of estimating <5, a and /3 from a functional sample yi,y 2 , ■ ■ ■ ,yn 
reduces to estimating the set of real-valued parameters {dm, bm,m' '■ m,m' = 1,. ■ ■ ,M}. To this end, 

project first ..., andcj^,..., onto and define the M-dimensional vectors = {y^'^i ,..., 
and sf ^ ..., through their entries y^'^^ = {yf, ipm) and = (cr^, ipm), where (•, •) denotes 

the inner product in I? and ^ stands for the transpose of vectors and matrices. Using (|1.2|) it follows that 


sf ^ = d + Ay£\ + 


(3.3) 


where d = {di, ..., with dm = {d, (fm) and A and B are M x M matrices whose (m, m')th entries are 
given by Om,m' and bm,m'- With this multivariate formulation of the volatility equation, iterating p.3| ) yields 

oo oo 

sf ^ = E + E AyS, ( 3 . 4 ) 

e=i £=i 

where B*^ = I, the M x M identity matrix. Note that ( |3.4| ) provides a representation for the projections of the 
unobservable volatilities in terms of the observations. Define fhe generic equafion 

OO OO 

f ’ (® = E tf-'d + y; B'-'Ayj!’,, (3.5) 

i=i e=i 

where 0 = (d"'", A^, B^)"'' G 0 C If is clear fhaf = s^^^(0), where 9 = (d'*', A"*", B 

is fhe frue paramefer of fhe fGARCH(l,l) process (yi: i G Z). Since fhis can always be achieved by an 
appropriate scaling, if is assumed wifhouf loss of generalify fhaf 


1 ^ ^ [ 0 ) !]• 


(3.6) 


If Tj = a{s£: i < i) denofes fhe ir-algebra generafed by fhe all innovations up fo i, fhen ( |3.6| ) implies fhaf 

j'O) ('2') 

E[y) \3^i] = ^ and hence the least square estimator of 9 is given as the smallest value of the criterion 
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function Yl'i=i{yT^ ~ Since only yi,.. .yn are observed, the actual parameter 

estimator is based on a truncated version of p.5| ), namely on 

£=i e=i 

This leads to the least squares estimator 

h = argminj 5„(0) = - sf^(0)}^{yf^ - sf^(0)}: 0 G ©j, 

with 6^ = (dj, AJ, B^)^. The main result of this section is the strong consistency of Some guarantees 
to ensure the identifiability of 6 are needed and collected next. Let || • || denote the Euclidean norm of vectors 
and matrices. 


Assumption 3.1. It is assumed that 

( 2 ) 

(A1) y^ is not measurable with respect to To; 

{A2) A is nonsingular and ||B|| < 1; 

(A3) & is a compact set and 6 is in the interior of 0; and 

(A4) there are 0 < ci and C 2 < 1 such that ci < |det(A)| and ||B II < C 2 for all 9 = (d^,A^,B^)T G 0. 


.(2) 


Part (Al) of Assumption 3.1 means hat y![ cannot be predicted almost surely from its past. Parts (A3) 
and (A4) on the parameter space © contain standard regularity conditions. 


Theorem 3.1. Assume that ( |1.1| )-( [L4| ) and p.2| ) hold with 0 = 1, that E[||eo|| 2 ] (3.6l is satisfied. 

Then is strongly consistent for 0 under Assumption . 


3.1 


that is, 


0n 
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a.s. 


as n ^ oo. 


To establish the asymptotic normality of 9^, introduce the quantities 


Ho=E 


d^o\G) 


and 


89 


Qo — E 


Jn — E 


dso\G)Y f ds'^o\^) 




89 


89 


The matrices Hq and Qo exist under the assumptions of Theorem 3.1 If E[||(Tq|||] < oo and E[||eQ|||] < oo, 
then Jo is well defined. Following the literature on the asymptotic normality of M-estimators, it is further 
assumed that 

Qo is a non-singular matrix. (3.7) 


7 















Theorem 3.2. Assume that the conditions ofTheorem \3.l\ are satisfied, p.7| ) holds and II 9 ] < 00 . Then, 
as n ^ 00 , 

where N is a 2M^ + M-dimensional normal vector with E[N] =0 and E[NN''~] = Qg ^Hg JqHoQq 


The proofs of Theorem 3.1 and 3.2 are given in Section]^ Typical choices for the orthonormal class of 
functions include the common Fourier and i?-spline bases, but other choices such as wavelets may be 
entertained as well. If the finite-dimensionality condition ( |3.2| ) is not satisfied, then <5, a and /3 can typically 
still be well approximated by finite-dimensional functions assuming that M is sufficiently large. While for 
theoretical considerations M should grow with the sample size, it is worthwhile noting that in practice often 
a small choice of M (less than, say, 5) will already work reasonably well. This aspect was investigated in the 
simulations reported in Aue et al. (2015) and is also reported elsewhere in the functional data literature (see 
Horvath and Kokoszka, 2012). 

A different route for utilizing the estimation procedure that leads to a small class is to follow the 
idea of functional principal components analysis (see Horvath and Kokoszka, 2012). For example, it can be 
proposed to use the eigenfunctions of the covariance kernel 


D{t,s) = Cov(yo(f),yo(s)), f, s E [0,1], 


as a basis, so that a few large eigenvalues and their corresponding eigenfunctions capture the most important 
directions of randomness in ..., and hence in the unobservable volatilities ci^,..., cr^. Since D{t, s) is 
unknown, the estimation is based on the empirical covariance kernel 

Dn(t,s) = (3.8) 

i=l 

where yi^^ = ^ vf- The spectral decomposition of Dn{t, s) gives rise to the ordered empirical eigen¬ 
values Al > ... > An and the corresponding empirical eigenfunctions (^1 ,... Therefore, may 
be replaced with the class of estimated eigenfunctions = {(pi ,..., (pu}, with appropriately chosen M. 
The following result shows that consistency can be established also for the functional principal components 
approach to estimating the fGARCH(l,l) process. 


is retained if the random class 

used in place of a deterministic class 


Corollary 3.1. If f K[yQ{t)]dt < 00 , then the result of Theorem 


3.1 


The proof of Corollary |3.1|is given in Section 





4 Empirical results 

4.1 Simulations 


Figure 4.1: Five consecutive fGARCH(l,l) observations (read row-wise starting from the top-left) generated 
from model ( |4.1| )-( |43] ). 







In order to investigate the performance of the least squares procedure in finite samples, functional data 
was generated from the fGARCH(l,l) process : z G Z) whose volatility function ( |1.2| ) is specified fhrough 
fhe consfanf infercepf funcfion 6 given by 

d(t)=0.01, tG[0,l], (4.1) 

and fhe operafors a and /? defined fhrough fheir kernel funcfions 

a(t, s) = (3{t, s) = 12t(l — f)s(l — s), t,s € [0,1]. (4.2) 
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The innovations (e*: i G Z) appearing in (|1.1|) and (|1.2|) are defined as 


£i{t) = 


Vbp /2400i\ 

2200f 


f G [0,1], 


(4.3) 


where {Bi : f G Z) are independent and identically distributed standard Brownian motions. These were 
constructed from the equi-spaced grid j/285, j = 1,..., 285. Lengthy but elementary calculations show 
that {TA) holds in case of ( |4.1[ )-( |43| ) and therefore {yi: i £ Z) constitutes a strictly stationary process. The 
recursion in ( |1.2| ) was initialized with af = 6 and the first 1000 simulated curves were discarded as burn-in 
values to get close to the stationary solution. Figure |4.1| displays the first five simulated functions after the 
burn-in. 

Random samples of size n = 300, 600 and 1200 were obtained from the data generating process in 
equations ( |4.1[ )-( |43| ). Note that in this setting a{t,s) = /3{t,s) = OA - ipi{t)(pi{s) with the normalized 
function ipi given by 

(pi{t) = ■\/fWf(l —t), t £ [0,1]. 


Equation ( |3.2| ) shows then that the operator-related parameters to estimate are an = bn = 0.4. Note that 
the intercept function 6 = 0.01 is not in the span of ipi and therefore does not satisfy ( |3.1[ ), so that the 
corresponding least squares estimate becomes a proxy for di = {6, ifi) = v^/600 Ri .009. The estimation 
procedure was run 1000 times with M = 1 yielding the outcomes reported in Table |4.l[ 

An alternative method for estimating S can be obtained as follows. Observing that if a and /d are asymp¬ 
totically consistent estimators for a and fi in the L^-sense, then 


- (a + i5)y^n^ 

is an asymptotically consistent estimator for 6, where yn'^ = ^ Vi- present simulation setting, 

there was little difference between 5 = diip and 5, so only results for the first choice are presented here. 


Table 4.1: Estimates di, dn and bn with M = 1 and = \/^f(l — t) in model ( |4.1[ )-( |431 ) for different 
sample sizes re, with sample standard deviations given in brackets. The row labeled by oo shows the population 
values. 


re 

di 

dll 

bn 

300 

0.013 (0.003) 

0.420 (0.058) 

0.306 (0.086) 

600 

0.011 (0.002) 

0.412 (0.042) 

0.344 (0.064) 

1200 

0.010 (0.001) 

0.408 (0.028) 

0.369 (0.045) 

oo 

0.009 

0.400 

0.400 


To illustrate the estimation method using estimated functional principal components, whose large-sample 
behavior is given in Corollary |3.1[ re = 1000 functional observations were generated from the fGARCH(l,l) 
model (|4.1|)-(|43|). The largest eigenvalue of Ilipoo. the empirical covariance kernel defined in (^), explains 
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Figure 4.2: Plots of a{t,s) = /3{t,s) of ( |4.2| ) (top panel) and corresponding estimates diooo(i)'S) and 
/3iooo(f; s) (lower panel) based on the data generating process (|4l])-(|43|). 



about 70% of the variability in the squared functions yf, while the second-largest eigenvalue only accounts 
for 4%. First, the estimation procedure was applied using only the first empirical eigenfunction ipi. The 
resulting estimated kernels diooo(f)'S) and /3iooo(i,s) are shown in Figure 4.2 along with the population 
kernels a(t, s) = I3{t, s) given in ( |4.3| ). The experiment was then repeated using two or more eigenfunction 
but no qualitative improvement in the shape of the estimators aiooo(f) s) and /3iooo(f) ■s) was found. 
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4.2 Application 


Figure 4.3: Plots of 5-minute log-returns of the SPY ETF from July 21 and 24—27, 2006 (read row-wise 
starting from the top-left). 







The data example eonsidered in this subseetion is on intra-day log-returns of an exehange-traded fund 
(ETF), namely the popular SPDR S&P 500 EDF (SPY) designed to traek the S&P 500 index. Eet then pi{t) 
denote the value of the SPY ETF on day i at time t. Following Cyree et al. (2004) one ean set up the intra-day 
log-returns 

yi{t) = logpi{t) - logpi{t - h), 

where /i is a time window typieally eorresponding to 1, 5, or 15 minutes. In the following, only h = 5 and 
therefore 5-minute log-returns are eonsidered. This is done for eonvenienee beeause the priee of the SPY ETF 
is reeorded in five-minute resolution, for example, at the website https : //quantquote . com, and the 
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5-minute log-return proeess (yj) becomes easily computable. Since the SPY trades from 9:30am to 4:00pm, 
there are 78 measurements per day. The resulting high-dimensionality of the intra-day observations make the 
application of any multivariate ARCH or GARCH model impractical and a functional alternative is suggested 
in the following. Note that the volatility of the log-returns on day i at time t is then represented by 

= Var(yi(f)|Ti_i), 

where 3'i-i denotes the the available information until the end of day i — 1. Figure [43] shows the graphs of 
the 5-minute log-returns of the SPY data for five consecutive trading days in July 2006. 

Figure 4.4: Estimates for intercept function 6 (top), and kernels a{t, s) (bottom left) and /3(f, s) (bottom right) 
based on the SPY ETF data. 
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1.0 


0.0 


The full data set considered here covers the trading days between January 2, 2003 and December 30, 
2006. SPY is not traded on weekends and bank holidays. There were eight half trading days during this four 
year span and due to possibly different market behavior these eight curves were excluded from the analysis, 
resulting in the fitting of an fGARCH(l,l) process to n = 999 functional observations. The least squares 
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estimation procedure was carried out using consisting of the empirical eigenfunctions associated with 


the M largest eigenvalues of the covariance operator -D 999 . Figure 4.4 shows the estimates for the intercept 


function 5 and integral kernels corresponding to the operators a and /3 for the case M = 1. Note that the 
largest eigenvalue explains 56% of the variation. As in the simulated data example, increasing the number 
of eigenfunctions used, did not qualitatively improve the estimators. It should be noted that 6 has a global 


maximum shortly after trading opens in the morning and a further local maximum at the end of the day, 
corroborating similar findings in other papers. 

5 Proofs 

5.1 Proofs of Section |2] results 

Using (O', the volatility equation (| 1 . 2 [) can, for any t G [ 0 , 1 ], be written as 



(5.1) 


with integral kernel 


ji-iit, s) = a(t, s)£-_i(s) + /3(f, s). 

Iterating o backward yields 



and so on. For i £ Z, let Fj 0 denote the identity operator and, for f G Z and /c G N, let the random integral 
operators Fj ^ be defined by fhe equafion 


(Fj^fe x){t) = si) 7 i_ 2 (si, S 2 ) • • • 7 i_fc(sfc_i, Sk)x{sk)dsk ■ ■ • ds 2 dsi. 



Then, if follows fhaf, for any m G N, 



(5.2) 


If can be seen from fhe definition of Fj ^ fhaf 


fij/c ^*,1 ° Ti—1,1 O ■ ■ ■ O Fj_/j_|_]^ , 


(5.3) 
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where o denotes the composition of operators. The foregoing identifies 






(5.4) 


k=0 


as candidate for the nonanticipative solution of ( |1.1| ) and ( |1.2| ). It remains to be shown that the sum on the 
right-hand side of equation ( |5.4| ) is well defined with probability one. This will be verified in the following. 


Proof of Theorem [Zi] (i) Consider the convergence of the sum in ( |5.4| ) in the space of square integrable 
functions on [0,1]. Since 

(Ti^ix){t) = j s)x(s)ds 

defines an integral operator in it follows from Riesz and Sz.-Nagy (1990, p. 148) that 


< llTi-lIb, 


(5.5) 


where || • || 3 \f denotes the operator norm on Using representation (5.31 and inequality (|5.5|) leads to 


IIW — 


<ni 

1=1 


yi-Pn- 


(5.6) 


(The usual arrangement nie 0 = 1 applies here and in the following.) Since || 7 r ||2 are independent and 
identically distributed random variables, the strong law of large numbers implies that 

-I ^ 




'i-r||2 


IE[log||7o|h 


a.s. 


(5.7) 


1=1 


Hence, in view of ( |2.1| ), 


p\ =1 


fc =0 


It is easy to see that that the infinite sum on the right side of ( |5.4| ) satisfies ( |5.1| ) and hence ( |1.1[ ) and ( |1.2| ). 
To prove the uniqueness of the solution to (|1.1|) and \\.2) assume that af is another such solution. Using the 


backward recursions as in ( |5.2| ) shows that, for all m G N, 

m —1 
~ ^ ^ ^ 


;^2 

i—m‘ 


(5.8) 


k=0 


Hence, for all m G N, 




k=0 


f2r.kS < Er.,** 


k=m 




(5.9) 


Assumption (|2.1[) implies for the first norm on the right-hand side of the latter equation that, as m —)■ oo. 


^ ||rj^fc|| 3 vf||(i ||2 —0 a.s. 

k=m ^ k=m 
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Similarly, 


m ||2 — mib 0 ) 

which follows since ||rj^m||w 0 and, due to the weak stationarity of {&i'. i G Z), the sequence 

(lld^-mlb: * G Z) is bounded in probability. Therefore 


P 


which proves the first part of the theorem, 
(ii) Note that 


cr,: - 




fc =0 


= 0 ^ = 1 , 


oo k 


Ikolb < X] '^Ib < ll'^lb X] l|ro,fc||M < Ibib X] 11 llTo-^lb- 

fc=0 k=0 k=0£=l 

Consequently Minkowski’s inequality (cf. Hardy et ah, 1959, pp. 24-26) implies that, for k' > 1, 


OO / / k \ i/\ Xju oo 

Nko^ll2])'^"<ll<5|bE®^n 11^0-^112 <ll<^lbE 

fc = 0 ^ 


^]y 


and, for 0 < 1 / < 1, 

OO 

E[lkoll 2 l<ll«ll 2 E(®[ll^«ll 2 l)‘ 

The second part of the theorem is proved. 


k=0 


□ 


Proof of Corollary \2.1\ (i) The Bernoulli shift representation in (^1 is an immediate consequence of (5.4i. 

(ii) Ergodicity is implied by the work of Stout (1974). 

(iii) The statement follows immediately from (iv). 

(iv) For j G N, let 

liAi-e-jit, s) = a{t, + /3(f, s) 

and, for k > i, let the operators T* ^ ^ be defined through 


(ri,£,kx){t) = j ■ 


7i_l(f, Si)7i_2(si, S 2 ) • • • 7i-t!-l(s£-2, S£-l) 

22 7i,r,i— 1) Si) • • • 7i,£,i— A: (Sfc— 1 , • • • dsi- 


Setting 


it can be seen that 


i-i 


— 

— 




fc =0 


k=i 


E[lk.'- 2 Tf,,ll 2 ] < 2 "E 


k=l 


+ 2 ‘'E 




k=i 
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It follows from the proof of Theorem|2.1|that 


E 




k=l 


2J 


< 


k=i 

o 


if ly > 1, 


if 0 < < 1. 


k=i 


Observing that 


E 




k=e 


2J 


= E 




k=£ 


2J 


and that -E[|| 7 o|| 2 ] < 1 by assumption, the proof of (^i is complete. □ 

Proof of Theorem [27^ (i) Recall relations ( |5.2| )-( [54l ). It follows from the definition of Fj ^ that 

k 

l|ri,fc <5||e < ll'^lle n 

with 7 j(f) = f jj(t,s)ds. Hence 


£=1 




k=0 


< Pile 


oo k 

En 

k=0£=1 


yi-£\\e 


The strong law of large numbers yields that 

k 

^E^°sll7i-^lle ^IE[log||7o||f 


a.s., 


£=1 


implying immediately that, under assumption ( |2.5 1 

^ OO 

p\ Eri.fc'^ee[o,i][ = 1. 


k=0 


Thus the existence of a solution to ( |1.1[ ) and ( |1.2| ) is proven. Assume that {a'f : f G Z) is an other solution to 
( |1.1| ) and ( |1.2| ) in S[0,1]. Since ( |5.8| ) holds, if follows as in (^1 that, for all m G N, 


CTi - 


k=0 




k=m 


“r i.m 


t,m ^i-m\\Q 


If p.5|) is satisfied, then, as m —)■ oo, 

OO 

E 


k=m 


< Pile 


k=m 

using that (1/m) log IlFj^mlle E[log pop] < 0 with probability one. By the assumed weak stationarity 
of the function series (di: z G Z), the univariate sequence (p/_^p: z G Z) is bounded in probability and 


therefore 











































Consequently, 


P 


(Ji - 




k=0 


= 0 = 1 , 


and the uniqueness of the solution to (|1.1|) and (|1.2|) is established. 


(ii) The proof is similar to the proof of part (ii) of Theorem 2.1 


□ 


Proof of Corollary \2.2\ The proof follows along the line of arguments used to establish Remark 2.1 and the 
details are thus omitted. □ 

5.2 Proofs of Section |3] results 

Proof of Theorem [J7] First observe that, by ([T^. {vh Prn) = (o-fef, Pm) = (<7*^, Pm) + Pm)- 

Due to the independence of Sj and ai,yi-i, £ £ N, (|3.6|) and the stationarity of {yi\ i £ Z) imply that 


E[(yf - VCyf - sf)] = E - 1 ), Pm)^ + Mil), 


.( 2 ) 


M 


(5.10) 


m=l 


where 


M(0) = 0 e 0. 


Since M(0) > 0, M(0) = 0 and the first term on the right-hand side of ( |5.10| ) is constant as a function of 9, 
it suffices to show that 

M{e)>0 if 9^9. (5.11) 


Assume now that ( |5.11| ) does not hold, so that there is 9^ 9, 9^, £ & such that M{9^) = 0. But this means 

that = Sq^^( 0*) with probability one. Using p.4|) and (|3.5|), it can be concluded that 


,( 2 ) 


B'-M + B^M, + B^l A,yi, 

r=i e=i e=i e=i 

with 0* = (d*, A*, B,„). If £* is the smallest integer such that B^*“^ A B^*“^ A*, then 

OO OO OO 

(B^‘-i A - Bf-1 A*)yi2j. = ^ B^-M - ^ B^M* + ^ (B^'^ A - B^l A*)y 


(5.12) 


( 2 ) 

-I 


l=\ 


l=\ 




( 2 ) 

and therefore y_^* is measurable with respect to T-r*-! which contradicts part (Al) of Assumption 


3.1 


due 


to stationarity. Hence B^A = B^ A* for all £ > 0 resulting in A = A* and consequently B^“^A = 
B^“^A, in turn implying that (I—B)~^A = (I—B*)^^ A. Thus B = B*. Since A = A* and B = B*, 
( |5.12| ) shows that B^“^d = B^“^d* and hence (I — B)^^d = (I — B)“^d*, completing the 

proof of d = d*. By contradiction, (|5.11|) is established. 
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The ergodic theorem (cf. Breiman, 1968) yields that, for all 0 G 0, 


. n M 

Zn{0) = - ^ {yf^ - s®( 0 )}^{yf^ - [Woi4 - +M{0 a.s. 


2 = 2 


m=l 


( 2 ) 

as n —)■ cx). Observing that the derivatives of Sq ' have a uniformly bounded expected value on 0 G 0, 
standard arguments (see, for example, the proof of the uniform law of large numbers in Ferguson, 1996) show 
that 


sup 

ee© 


Z„ 


M 




m=l 


0 a.s. 


Using (3.51 and the assumption that ||B|| < p < 1 on 0 gives 


n sup\Zn{0) — Zn{0)\ = 0{1) a.s., 

0 e© 

where Zn{0) is defined as Zn{0) but using s\^\0) in place of sp^(0). Thus, 


sup 

0e© 


M 


Zn{0) - { E [{^{4 - 1), + M{0) ) ^ 0 a.s. 

The results in ( |5.11| ) and ( |5.14| ) imply the consistency in Theorem 3.1 via standard arguments. 
Proof of Theorem [!0] Note first that 


dZn{0, 

de 


= 0 . 


Reasoning as on page 466 of Seber and Lee (2003) yields 


and 


where 


dZn{0) 

de 


d^M0) 2 


n 


E 

i=2 




de 


” ' ds‘f\0)Y f d4^\0) 


de'^ n ^ V de 

~ 1=2 


E 


de 




i=2 


^ / fP s(2) (g'\\ 

R.(® = -I E ( J'7„ ) {(rf- *’"•) - : j. = 1,..., + Af L 


2,m V 

— V de.de^^ 

m=l ^ 


with 0 = ( 01 ,.. .,02m^+mV and sf^(0) = (4? (^), • • ■, 4^li{0)y ■ Let 


Q(0) = E 


ds^Q\0)Y (dy^\0) 


de 


de 


+ E[Ro(0)]. 


(5.13) 


(5.14) 

□ 

(5.15) 


It is easy to see that 


Q{0) is coninuous on 0 and Qo = Q{e). 


(5.16) 
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Hence the ergodic theorem (see again the proof of the uniform law of large numbers in Ferguson, 1996) 
implies that 

d^ZniO) 


sup 

0 e© 


89^ 


- 2Q(0 


0 a.s. 


(5.17) 


The central limit theorem for orthogonal martingale differences in Billingsley (1968) yields moreover that 

-idZn{e) D 


n- 


2 89 


N, 


(5.18) 


where N is a 2M^ + M-dimensional normal random vector with E[N] = 0 and E[NN^] = Hq JoHq. 


Arguing along the lines of the justification of ( |5.13[ ) one can verify that 

= 0(1) a.s. 


and 


n sup 
0 e© 


n sup 
e&@ 


8Zn{9) 

8Zn{9) 

89 

89 


8^Zn{9) 

8f 

89^ 


= 0(1) a.s. 


Combining ( |5.15 1 with the coordinate-wise mean value theorem for vectors leads to 

dZn{9) 


89 


= V„( 0 „- 0 ), 


(5.19) 


(5.20) 


(5.21) 


with some matrix V„. From ( 5.16| ), ( |5.17[ ) and ( 5.20 1 , it can be concluded that the matrix V„ satisfies 

V, ^ 2Qo. (5.22) 


Displays ( |5.18[ ) and ( |5.19| ) also yield fhaf 


1 8Zn{9) D ~ 

"2^ir ^ 


(5.23) 


where N is defined in ( |5.18| ). Puffing fogefher ( |5.21| )-( [5.23[ ), fhe asympfofic normalify of fhe esfimafor in 
Theorem l3.2l is esfablished. □ 


Proof of Corollary |3 . 1 \ The approximabilify esfablished in Corollary 2.1 implies fhaf \\Dn — D\\ —)• 0 in 
probabilify and Iherefore 

/V p 

max ||(^m - Cm<Pm|| 0, (n->(X)), 

l<m<M 

where fhe are random signs, assuming fhaf Ai > • • • > Am > ^M+i are fhe eigenvalues of D in 

remains frue 


decreasing order (see Horvafh and Kokoszka, 2012). Hence fhe consisfency of Theorem 3.1 


when fhe funcfions in $m = {<Pi,... j Pm} are used fo sef up ( |3.2 1 . 


□ 
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